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Abstract 

Information functions are used to find the optimum ability 
levels and maximum contributions to information for estimating item 
parameters in three commonly used logistic item response models. 

For the three and two parameter logistic models, examinees who 
contribute maximally to the estimation of item difficulty contribute 
little to the estimation of item discrimination. This suggests that 
in applications that depend heavily upon the veracity of individual 
item parameter estimates (e.g. adaptive testing or test 
construction), better item calibration results may be obtained (^or 
fixed sample sizes) from examinee calibration samples in which 
ability is widely dispersed. 

Keywords : Item response theory 

IRT item parameters 
Optimum examinees 
Information functions 
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Introduction 

The success of applications in Item Response Theory (IRT) 
depends upon the accuracy with which individual item parameters can 
be estimated. This dependence is especially important for those 
applications that depend heavily upon the veracity of individual 
item parameter estimates in contexts perhaps unrelated to the 
calibration setting in which estimates were obtained. Two recent 
examples of such applications are adaptive testing (see, for 
example, Lord (1980, chap. 10); Stocking (1988)) and IRT-based test 
development (see, for example, van der Linden & Boekkooi-Timminga 
(1987)). 

Typically, calibration samples are selected with an eye to 
convenience as random subsamples from a larger set of data (see, for 
example, Cook, Petersen, & Stocking (1983)). It may be possible to 
obtain better, i.e., more accurate, item parameter estimates if the 
planning for calibration sample selection explicitly includes 
considerations of the accuracy of the resulting item parameter 
estimates. This idea was suggested, based on the analogy between 
item parameter estimation and estimation problems in regression, 20 
years ago by Lord (1968). More recently, Wingersky and Lord (1984) 
provide theoretical support. 
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One approach to issues of the accuracy of item parameter 
estimation considers the asymptotic expected covariance matrix. 

Lord (1980, p. 191) presents formulas from which this matrix can be 
derived for the three -parameter logistic (3 PL) model in the finite 
sample case with known abilities. Thissen and Wainer (1982) 
consider the properties of the asymptotic covariance matrix for item 
parameters when a particular distribution of ability is assumed in 
the calibration example. Lord and Wingersky (1985) consider the 
asymptotic properties of the covariance matrix for the 3PL model 
when both item parameters and abilities are estimated jointly by 
maximum likelihood. 

In this paper we explore the relationship between examinee 
ability and the accuracy of maximum likelihood item parameter 
estimation in terms of the expected (Fisher) information (Kendall 6c 
Stuart, 1979, p. 10). The advantage to this approach is that it is 
possible to study directly the relationship between examinee ability 
and contribution to information in some detail and with some 
surprising results. This theoretical examination depends upon the 
assumption that examinee abilities, as well as some item parameters, 
are known. However, the results can be translated into general 
guidelines for situations in which samples can be selected on the 
basis of some observed score for the purpose of obtaining accurate 
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parameter estimates for a collection of items whose properties are 
imperfectly known. 

The Theory 

Suppose the probablility of a correct response to an item is 
specified by a logistic function, either one parameter logistic 
(IPL), two parameter logistic (2PL) or three parameter logistic 
(3PL) . SupT-ose further that estimates of the item parameters are 
obtained by maximum likelihood. If the model fits the data, and 
true abilities are known, then formulas exist for the amount of 
information in the sample for estimating item parameters (Lord, 

1980, eqs . 12-8 through 12-13)). 

Of course, the model never fits any set of real data, and true 
abilities are never known. Therefore, the information computed from 
the Lord formulas is an estimate of the maximum possible 
information, and will be larger than what can be realistically 
obtained (Mislevy & Sheehan, 1987). Nevertheless, an examination of 
what the theory predicts may be useful in planning a calibration 
sample to estimate item parameters accurately. 

If the probability of a correct response by examinee a to an 

item is P , the log of the likelihood of observed responses to the 
Si 

item for N examinees is 



O 
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N 



i = S [u inP + (1 



a a 



u^)inQ^] 



a=l 



where u = 0 if the response in incorrect, u == 1 if the response 
a a 

is correct, and Q = 1 - P . The maximum likelihood estimates of 
^a a 

each item parameter \ located at a point where the partial 

derivatives of the log likelihood are zero. The expected second 
partial derivatives of the log likelihood can be computed at this 
point. (The inverse of the negative of this matrix is the 
asymptotic variance/covariance matrix used by Thissen and Wainer 
(1982).) The negative of a diagonal element of the matrix of 
expected second partial derivatives is referred to as the 
information in the sample for estimating an item parameter. For any 
item parameter this information function has the form (Lord, 

1980, equations 12-8 to 12-13) 



The information is composed of individual additive contributions 
from each examinee , 
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_1 , ap 2 
“ PQ ax ^ 



( 1 ) 



where the subscript a has been dropped for convenience. By 
examining the properties of an individual contribution as a function 
of ability, we can determine what values of ability provide the most 
(and least) information for estimating an item parameter, assuming 
other item parameters are fixed. 

Results 

The 3 PL Model 

The 3PL item response function is 



P 



c 



+ 



1 



1 > c 

^ ^-Da(^?-b) 
+ e 



where 

a is a function of the slope of the item response function 
in the neighborhood of the item difficulty; 

b characterizes the difficulty or location of the Item on 
the ability continuum; 

c is the lower bound of probabilities of correct response, 
even from low ability examinees; 

0 is examinee ability; 
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D is a scaling constant commonly used with the value of 1.7. 

The partial derivatives of P with respect to the item 
parameters are given by Lord (1980, Equation 12-2). Substituting 
the derivative with respect to c into Equation 1 gives an examinee's 
contribution to the information for estimating c as 



i 

cc 



(1 



1 Q 

,2 P 
- c) 



( 2 ) 



Values of ability that give local minima and maxima of Equation 2 
are found where 



di 

cc 

de 



-Da 




4 Q(P - C) = 0 
P 



This derivative is zero when Q * 0, in which case ^ - -foo, and i 

cc 

is zero. It is also zero when P = c, in which case 0 * -<» and 
i = ~7Z ^ V . This latter value forms an upper asymptote to 

the amount of information from all ability levels. The higher the 
value of c, the more examinees, even optimal examinees, are required 
for estimation of c. 
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The information for estimating b is obtained by substituting 
the derivative of P with respect to b into Equation I, giving 



r^2 2 

D a 



Sb 2 

(1 - c) 



2 Q 



(3) 



with partial derivative 



di 



bb 



(1 - c)^ p2 



(4) 



A root of this derivative is found where Q - 0, in which case 
^ « +C30 and i, , * 0. A second root is found when P * c, where 

DO 

$=. .CO and L , « 0. Examinees with abilities far away from the 
bb 

item's location are useless for estimating that location. A final 

2 

root of this derivative is found where -2P + P + c=*0. The root 

of this latter quantity, P*, where c < P* < 1, is 



p* 



1 Vl 8c 



The optimal ability is then 



1 P* - c 



1 - p* 



( 5 ) 
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and 



bb 



2 

D a 

(1 . c) 



2 (P* ■ c) 



2 Q* 

p* 



( 6 ) 



where Q* === 1 - P*. 

The optimal ability level is at - b when c * 0. When 
0 < c < 1, the optimal ability level is greater than b by an amount 
that depends upon both a and c. For a fixed value of c, higher 
values of a mean that the optimum location is closer to but still 
greater than b. Since P* is a function of c alone, the maximum 
amount of information from an examinee for estimating b, Equation 6, 
depends only on a and c, and for fixed c, is proportional to the 
square of a. 

By analagous computations, the information from each examinee 
(Equation 1) for estimating a for the 3PL is 



i 

aa 




b)" (P - c) 



2 Q 



(7) 



with partial derivative 
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di 



aa 



(9 - bUP - c) Q 



89 



[((? - b) 



Da 



(1 - c)‘ 



(1 - c) 



(-2P 



+ P + c) 

( 8 ) 



+ 2P] 



Three roots of Equation 8 are at 9 “ b, 9 - -®, and ^ = +<». For 

all three of these roots, i =0. Low and high ability 

aa 

examinees, as well as examinees whose abilities may be close to 
optimum for estimating b are of no use in the estimation of a. The 
location of optimal abilities is found from the last factor in 
Equation 8. This expression is zero when the optimal ability 



e-k-k = b + 



2(1 - c)?** 
Da(2P**^ - P** - c) 



(9) 



The optimal ability is not given explicitly by this expression, but 
values of 9"^'^ can be found by numerical methods. The top panel of 

Figure 1 shows P plotted as a function of ^ - b, from Equation 9. 

1 2 

When P ^ b ^ when P - 1, 9 ’ ^ 

suggests that these values form upper and lower bounds respectively 

for two regions in which the optimal ability might be found: 

1 2 

^ < b " ~ and ^ > b + ~ . 

Da Da 
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Insert Figure 1 about here 



Numerical Examples 

Equations 2, 3, and 7 are plotted as functions of ability in 
the top panel of Figure 2 for an item with a 1, b « 0, and c * .2. 
Additive contributions to information for estimating a or b from 
examinees at various ability levels must be read from the right-hand 
scale; those for estimating c must be read from the left-hand scale. 
Figure 2 demonstrates graphically what we have learned analytically: 
1) the contribution of examinees to the information available for 
estimating item difficulty is asymetric around b, with higher 
ability examinees contributing more information; 2) examinees who 
are most informative in the estimation of item difficulty are of 
little use in estimating item discrimination; 3) examinees who do 
contribute to the estimation of discrimination are asyraetrically 
distributed around the item difficulty; and 4) only low ability 
examinees contribute much information for the estimation of c, and 
there is a limit to this contribution. 



Insert Figure 2 about here 
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The effects of different values of a, for fixed b and c, on the 
additive contribution of examinees to information for estimating a 
are shown in the top left panel of Figure 3. Each examinee 
contributes substantially more information to the estimation of a if 
the value of a is low rather than high. More lower ability 
examinees than higher ability examinees are required to obtain a 
given amount of information for the estimation of a, regardless of 
the value of a, since lower ability examinees contribute less 
information than higher ability examinees. 



Insert Figure 3 about here 

The middle left panel of Figure 3 shows the effects of different 
values of b, for fixed a and c, on the additive contribution of 
examinees to information for estimating b. Changing b shifts the 
location of optimal examinees, but not the maximum contribution. 

The effects of different c's on the additive contribution for 
estimating c, for fixed a and b, are shown in the bottom left panel 
of Figure 3. For lower ability levels, the higher the guessing 
parameter, the less the additive contribution for each examinee; 
more examinees are required to obtain a given amount of information 
about c. Regardless of the value of c, higher ability examinees do 
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not contribute much. However, they contribute more if c is higher 
than if c is lower . 

Numerical values of optimal abilities and maximal contributions 
to information (in parentheses) are presented for typical values of 
a and b for c = .2 in Table 1. Optimal abilities and maximal 
contributions for b come directly from Equations 5 and 6. Those for 
a come from the application of numerical methods to Equations 9 and 
7. 



Insert Table 1 about here 



The 2PL and IPL Models 

The 2PL item response function can be considered as a special 
case of the 3PL with c = 0. In this case, 



1 4 . ,-Da(^?-b) 

1 + e 

The IPL item response function is also a special case of the 3PL 
with c == 0 and constant a. The expression for P is identical to 
that for the 2PL immediately above, but the item discrimination is 
constant for all items. 
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The information for estimating b for the 2PL is algebraically 
the same as for the IPL: 



= D^a^PQ , (10) 

with derivative with respect to ability 

^ D^a^PQd - 2P) . 



These are obtained form Equations 3 and 4 for the 3PL by setting 

c = 0, Examinees with abilities of contribute nothing to the 

estimation of b in these models. Examinees with = b (P'^ = .5) 

2 2 

contribute the maximum amount of information, .25D a . The additive 
contribution from each examinee depends upon the square of the 
discrimination of the item. 

For the 2PL model, the information for estimating a is 
^aa " 



with derivative 
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3i 

aa 

ae 



= - b)PQ[Da(^ - b)(l - 2P) + 2] 



again obtained from the 3PL results (Equations 7 and 8) by setting 
c *= 0. As with the 3P^-, examinees with low and high abilities, as 
well as examinees whose abilities are optimal for estimating b are 
useless for estimating a. The location of optimal abilities is 
found by setting Da(^ - b)(l - 2P) + 2 =* 0 in which case the optimal 
ability is 



S'k'k 



Da(2P - 1) 



( 12 ) 



The bottom panel of Figure 1 shows P plotted as a function of ^ - b 

2 

for the 2PL in Equation 12. When P“0, and when 

2 

P=l, ^-b=~. As with the 3PL, there are two regions in which 
ua 

2 2 

Q'k'k might be found: ^ < b - ~ and ^ > b + ~ . In contrast to 

^ Da Da 

the 3PL, these regions are symmetric around ^ - b * 0. 

The additive contribution to estimating a for optimal examinees 
is 



i 

aa 



2 2 1 
.25D - b) - — . 

a 



(13) 
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Both terms in Equation 13 decrease for higher values of a since the 
optimal B is closer to the difficulty for increased a, given fixed 
b. In contrast with the 3PL, the contributions of examinees whose 
true abilities are symmetric around the item difficulty are 
identical . 

Numerical Examples 

Equation 10 for the 2PL and IPL and Equation 11 for the 2PL are 
plotted as fuctions of ability in the bottom panel of Figure 2 for 
an item with a ■= 1 and b = 0. Additive contributions to information 
for estimating a (for fixed b) or b (for fixed a) are higher in 
models that do not contain a guessing parameter. This is seen by 
comparing the two panels in Figure 2. 

The effects of different a values (with fixed b) on the 
additive contributions for estimating a in the 2PL (Equation 11) are 
shown in the top right panel of Figure 3. Examinees symmetrically 
located around the item difficulty contribute equally to 
information. As in the 3PL (top left panel), each examinee 
contributes more to information for estimating a when a is low. In 
contrast to the 3PL, examinee's additive contributions are higher 
when the model does not contain a guessing parameter (compare the 
two top panels of Figure 3). Equation 10 for the 2PL with fixed a 
or the IPL is plotted for different values of b in the middle right 
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panel of Figure 3. As with the 3PL (middle left panel), changing 
values of b shifts the location of optimal examinees but not the 
maximum contribution to information for estimating item difficulty. 

Numerical values of optimal abilities and maximum contributions 
to information (in parentheses) are presented for typical values of 
a and b for the 2PL and IPL in Table 2. Optimal abilities and 
maximum contributions for estimating b come directly from Equation 
10 and its consequences. Comparable data for information for 
estimating a for the 2PL comes from the application of numerical 
methods to Equation 12 and Equation 13. 



Insert Table 2 about here 



Discussion 

The focus of this paper has been to explore the contribution of 
examinees with different true abilities to the expected information 
available for estimating item parameters. This is not the same as 
an exploration of the accuracy with which item parameters may be 
estimated, but has the advantage of directly suggesting strategies 
of calibration sample selection. In terms of making statements 
about the accuracy with which item parameters are estimated, the 
analyses presented here are immediately relevant to the problem of 
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estimating a single item parameter, conditional on other item 
parameters being fixed at their true values. However, statements 
about the accuracy of item parameter estimates are usually made in a 
broader context in which all three item parameters must be estimated 
and a particular distribution of true ability may be assumed. Such 
considerations then take into account the covariances among the item 
parameters, and depend upon the assumed distribution of ability. 

The results need not hold for different ability distributions. 

As an example, the results presented here show that in both the 
2PL and 3PL, optimal examinees contribute more information for 
estimating item discrimination when that discrimination is low. 

This cannot be interpreted to moan that low a's are necessarily more 
accurately estimated than high a's, for a fixed sample size or a 
fixed distribution of ability. Thissen and Wainer (1982), using the 
expected asymptotic variance/covariance matrix and assuming a normal 
distribution of ability, show that, for fixed b, the asymptotic 
standard error of a decreases with a for the 2PL. However, for 
fixed b in the 3PL model, the asymptotic error of a is larger for 
both small and large values of a than it is for more moderate values 
of a. These results depend on the assumed distribution of ability 
as well as the covariances among the item parameters. 
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As an illustration of this phenomenon, Figure 4 shows the 
Thissen and Wainer asymptotic standard error of b plotted against b 
for the 3PL (solid curve). This is a partial reproduction of 
Thissen and Wainer' s Figure 1, with a = 1.5, c * 0., and a standard 
normal calibration sample of N * 2500, The results presented here 
suggest that a calibration sample with a larger spread of abilities 
will improve the accuracy of the estimation of high and low 
difficulties. Figure 4 shows the same information for the same item 
when a calibration sample of the same size is drawn from a normal 
distribution of ability with a variance of 4 (dashed line). 

Although the sample size is the same, the accuracy of estimation is 
much improved for the more extreme difficulties. 



Insert Figure 4 about here 




Given the knowledge of the location of optimal examinees for 
estimating the parameters for a single item in three different item 
response models presented here, what suggestions can we make for 
calibration sample selection that would aid practitioners who must 
jointly estimate the item parameters for many items simultaneously? 
If the practitioner chooses the IPL as the appropriate model, then 
examineees whose true ability is equal to item difficulty are most 

O .4 
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info rma t ive . If a collection of i terns to be cal ibra ted is thought 
to have- a broad spread of difficulties, based perhaps on the 
conventional proportions correct, then the distribution of true 
abilities in the calibration sample should also be broad. Such a 
sample could possibly be selected based on some available observed 
auxiliary information. Tf the range of abilities is too small, the 
sam.ple will provide information only for middle difficulty items; 
little information will be provided to estima*" ' the difficulty for 
easy and hard items. 

If the 2PL is the appropriate model, estimation of both a and b 
requires a wider range of true abilities than for the IPL. This is 
so because only examinees with ability not equal to b are 
informative in the estimation of a. If the range of abilities is 
too small, information for the estimation of difficulties for middle 
items may be provided, but information for estimating their 
discrimination may not be. Information for the estimation of 
discriminations for easy and hard items may be provided, but 
information for estimating their difficulties may not be. 

If the 3PL is the most appropriate model, only abilities well 
below the item difficulty are informative about c. Abilities below 
and above the difficulty are most informative about a and abilities 
slightly above the difficulty are most informative about b. Even if 
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all items are of equal difficulty, a normal ditribution of abilities 
centered slightly above the item difficulty may not provide much 
information for the estimation of all parameters simultaneously. If 
the items have a spread of difficulties, bette,3? results may be 
obtained by sampling all ability levels equally. Wingersky and Lord 
(1984) show that when item and ability parameters are estimated 
simultaneously, a sample of abilities drawn from a uniform 
distribution produces standard errors nearly as small as a sample of 
abilities four times as large dravnri from a bell-shaped distribution. 

Calibration samples, particularly for the more complex models, 
typically consist of several thousand examinees. Depending upon the 
nature of the collection of items to be calibrated, which can be 
roughly assessed through the use of conventional item statistics, 
such samples, although large, may not prove useful for estimating 
the paramters of all items. If the success of a particular 
application of IRT depends heavily on the veracity of item level 
data, it seems worthwhile to consider selecting more informative 
samples. 
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Table 1 

Optimum Abilities and Maximum Contributions (in Parenthesses) 
to Information for Estimating a or b Assuming Fixed Values 
for the Other Parameters for the 3PL Model with c = .2 



a 


b 


lower 
optimal 
^ for a 


upper 
optimal 
^ for a 


optimal 
^ for b 


.5 


-2 


-3.96 


.86 


-1.69 


(i = .58. 1.38) 

aa 


-1 


-2.96 


1.86 


- .69 


0 


-1.96 


2.86 


.31 


('bb = -'2) 


1 


- .96 


3.86 


1.31 


2 


.04 


4.86 


2.31 



1.0 


-2 


-2.98 


- .57 


1 

00 


(i = .14, .35) 


-1 


-1.98 


.43 


- .84 


aa 


0 


- .98 


1.43 


. 16 


(Sb = 


1 


.02 


2.43 


1.16 


2 


1.02 


3.43 


2.16 



1.5 


-2 


-2.65 


-1.05 


-1.90 


(i = .06, .15) 

aa 


-1 


-1.65 


-.05 


- .90 


0 


- .65 


.95 


.10 


(ibb = 1.11) 


1 


.35 


1.95 


1.10 


2 


1.35 


2.95 


2.10 



2.0 


-2 


-2.49 


-1.28 


-1.92 


(i = .04, .09) 


-1 


-1.49 


-.28 


- .92 


aa 


0 


- .49 


.72 


.08 


(ibb = 


1 


.51 


1.72 


1.08 


2 


1.51 


2.72 


2.08 



2.5 


-2 


-2.39 


-1.43 


-1.94 


(i = .02, .06) 


-1 


-1.39 


- .43 


- .94 


aa 


0 


-.39 


. 57 


.06 


(ibb = 


1 


.61 


1.57 


1.06 


2 


1.61 


2.57 


2.06 
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Table 2 

Optimum Abilities and Maximum Contributions (in Parenthesses) to 
Information for Estimating a or b Assuming Fixed Values for the 
Other Parameters for the 2PL and IPL Models with c = 0 



a 


b 


lower 
optimal 
B for a 


upper 
optimal 
0 for a 


optimal 
0 for b 


.5 


-2 


-4.82 


.82 


-2.00 


(i = 1.76, 1.76) 


-1 


-3.82 


1.82 


-1.00 


aa 


0 


-2.82 


2.82 


.00 




1 


-1.82 


3.82 


1.00 


2 


- .82 


4.82 


2.00 



1.0 


-2 


-3.41 


- . 59 


-2.00 


(i = .44, .44) 


-1 


-2.41 


.41 


-1.00 


aa 


0 


-1.41 


1.41 


.00 


(^bb= -^2) 


1 


- .41 


2.41 


1.00 


2 


.59 


3.41 


2.00 



1.5 


-2 


-2.94 


-1.06 


-2.00 


(i = .20, .20) 


-1 


-1.94 


- .06 


-1.00 


aa 


0 


- .94 


.94 


.00 


(ibb = 1.63) 


1 


.06 


1.94 


1.00 


2 


1.06 


2.94 


2.00 



2.0 

<^aa= 

(ibb =2.90) 


-2 

-1 

0 

1 

2 


-2.70 
-1.70 
- .70 
.30 
1.30 


-1.30 
- .30 
. 70 

1.70 

2.70 


-2.00 

-1.00 

.00 

1.00 

2.00 


2.5 


-2 


-2.56 


-1.44 


-2.00 


(i = .07, .07) 


-1 


-1.56 


- .44 


-1.00 


aa 


0 


- .56 


. 56 


.00 


(^bb = 


1 


.44 


1.56 


1.00 


2 


1.44 


2.56 


2.00 
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Figure 1. The top panel shows P as a function of 0 - b from Equation 
9 for the 3PL (solid curves). The bottom panel shows P as 
a function of 0 - b from Equation 12 for the 2PL (solid 
curves). 
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Figure 2. Contributions to information for estimating item parameters 
as functions of ability for the 3PL (top; a=1, b=0, c=.2), 
the 2PL and 1PL (boitom; a=1, b=0). 
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Figure 3. Contributions to information for estimating item parameters 
for the 3PL (left) and 2PL and 1PL (right). The effects 
of varying a conditional on fixed b=0 (and c=.2 for the 
3PL) are shown in the top row. The effects of varying b 
for fixed a=l (and c=.2 for the 3PL) are shown in the 
middle row. The effects of varying c for the 3PL condi- 
tional on a=1 and b=0 are shown in the bottom row. 



standard Error of b 





LEGEND 

N(0,1) distribution 
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Figure 4. Standard errors of estinnated itenn difficulties for the 3PL 
model, 0=1.5, c=0. A calibration sample of N^,500 is 
drawn from an N(0,1) distribution (solid) and from an 
N(0,4) distribution (dotted). 
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